Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning

In reinforcement learning (RL), the ability to utilize prior knowledge from previously solved tasks can allow agents quickly solve new problems. some cases, these problems may be approximately by composing solutions of primitive (task composition). Otherwise, used adjust reward function for a problem, in way that leaves optimal policy unchanged but enables quicker (reward shaping). this work, we develop general framework shaping and task composition entropy-regularized RL. To do so, derive an exact relation connecting soft value functions two RL with different dynamics. We show how derived leads result then generalize approach multiple validate theoretical contributions experiments showing lead faster various settings.